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ACCURATE TRANSLATION 

TITLE OF THE INVENTION 
IMAGE PROCESSING APPARATUS 

BACKGROUND OF THE INVENTION 
The present invention relates to an image 
5 processing apparatus for joining (restoring 

composition) a plurality of images taken to which one 
composition is divided with an overlap area where 
desired object exists or synthesizing a plurality of 
images taken with a different exposure, and more 
10 particularly to an image processing apparatus for 

extending a viewing angle of a joined image and its 
dynamic range of a synthesized image. 

In recent years, personal computers (hereinafter 
called PC) got much more capability and their price is 
15 reducing in accordance with improvement of manufacture 

technique, so that they have been widely used in many 
companies, education, and home. 

To input images to PC, an image is optically 
picked up from a film photographed by a conventional 
2 0 camera and is converted to an image signal to be input. 

In addition to camera, imaging apparatus such as 
a video camera for taking the image are used in various 
situations. Particularly, in a digital still camera, 
a film, which is used in the general camera, doesn't 
25 have to be, used. Instead, the image is converted to 

a digital signal and recorded in storage medium, which 
is magnetically or optically recordable, so that 
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the digital signal is input to the inputting device. 
Then f the image is reconstructed to be output to a 
display or a printer. As a result, developing process 
for films is unnecessary, and erasing and editing can 
5 be easily performed. 

Moreover, demand for digital still cameras has 
been rapidly increased for the purpose of using the 
Internet communication in accordance with an increase 
Internet users, reduce the price of the digital still 

10 camera. 

The digital still camera get a image of the object 
as an image signal by a solid state imaging device, 
such as CCD, using photoelectric conversion. However, 
since resolution and a dynamic range are low as 

15 compared with the ordinary film, the technique to make 

high resolution images and to extend a dynamic range of 
images have been strongly desired. 

One of methods for obtaining high resolution image 
is increasing the number of pixels of the imaging 

20 device itself. However, it is generally known that 

the cost of the imaging device rapidly rises with 
the increase in the number of pixels. 

The applicant of the present application proposed 
the technique of joining the images taken by a 

25 plurality of imaging device as described in Japanese 

Patent Application KOKAI Publication No. 6-141246, and 
the technique of divisionally photographing the object 
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by the movement of the camera so as to join a plurality 
of images obtained by one image pickup element. 

However, in general when the photographed image is 
subjected to influence of distortion due to the optical 
5 system, the image is distorted. If the images are 

joined by the technique of Japanese Patent Application 
KOKAI Publication No. 6-141246, the composition of the 
overlapped parts differs between the images, and there 
occurs a problem object is seen double in the join 
10 image. Also, due to the displacement of the points, 

serving as a reference for joining, the image is 
detected as if it were rotated though the image is not 
actually rotated. As a result, there occurs a problem 
in which the images are not joined well. 
15 To solve the above problems, the applicant of the 

present application proposed an image processing 
apparatus for compensating for influence of distortion 
comprising image correcting means for a geometrical 
correction as described in U.S. Patent 08/541,644 
20 (filing data: October 10, 1995). 

The structure of the image processing apparatus is 
shown in FIG. 18 . 

In the figure, each of image input sections la to 
lc of the image processing apparatus comprises an 
25 optical system 2, an image pickup section 3 such as CCD, 

and an A/D converter 4 . These image input sections are 
arranged to Capture different portions (positions) of 
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an object 5 to have the overlapping area. 

An output signal of each image pickup section 3 is 
digitized by the A/D converter 4 so as to be input to 
each of image correcting sections 17a to 17c. Each of 
5 the image correcting sections 17a to 17c reads 

photographing conditions such as a focus position when 
a image is taken and a characteristic parameter of the 
optical system so as to correct the distortion of the 
images taken by the image input sections la to lc . 

10 Next, in an image joining section 6, the images 

(serving as input signals), which are corrected by the 
image correcting sections 17a to 17c, are joined to be 
a wide-angle image as shown in FIG. 20. Then, the 
joined image is output to a motor 9, a printer 8 or 

15 a storage medium 9 . 

The image joining section 6 is realized by the 
structure as shown in FIG. 19. 

In this structure, the images a, b, and c are 
temporarily stored in a frame memory 10 respectively. 

20 Then, an amount of parallel movement SI and an 

amount of rotations Rl between the adjacent images 
(e.g., images a and b) are obtained by a shift 
detector 11a. Similarly, an amount of parallel 
movement S2 and an amount of rotations R2 between 

25 the images b and c are obtained by a shift detector lib. 

These amounts of parallel movement SI, S2, and 
amounts of rotations Rl and R2 are input to 
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interpolation calculators 12a and 12b f together with 
the images read from frame memories 10b, 10c. Thereby, 
the images whose positional relationship are corrected 
can be obtained. 
5 A coefficient setting device 13 sets coefficients 

Ca, Cb, and Cc of the respective images of FIG. 20 
such that the adjacent images are smoothly joined 
to each other. The pixel value of each image is 
multiplied by each of coefficients Ca, Cb, Cc by a 

10 multiplier 14. Then, the overlapping portion is added 

by an adder 15. 

FIG. 20 is a view showing the processing of the 
overlapping portion of the images to be joined. 

The image b rotates anticlockwise against the 

15 image a. The rotation of the image b and the amount of 

overlapping (or amount of parallel movement) are 
calculated by the shift detector 11. Also, as shown in 
FIG. 20, the pixel value of each image is multiplied by 
each of coefficients Ca, Cb, Cc so as to smoothly 

20 connect the images a and b, which are overlapped with 

each other. In this way, the image joining section 6 
outputs the image in which the plurality of images are 
joined with high resolution or a wide viewing angle are 
provided. 

25 Regarding to extend dynamic range of the imaging 

device, the applicant of the present application 
proposed the following technique in Japanese Patent 



Application KOKAI Publication No. 63-232591. 

Specifically, a plurality of images photographed 
with different exposure is synthesized so as to 
generate an image having a dynamic range, which is 
almost equal to the film. 

The above technique can be realized by structuring 
the image joining section 6 as shown in FIG. 22. 

FIG. 22 conceptually explains an example in which 
two images are synthesized. Even in a case of joining 
three or more images, the images are synthesized by the 
same process. 

Two images a and b are added to each other at an 
adder 21 to be stored in the frame memory 10. A linear 
converting section 22 reads out data of the frame 
memory 10. The linear converting section 22 calculates 
values corresponding to R, G, B values of incident 
light based on a look-up table so as to be input to a 
matrix circuit 23. The R, G, B values obtained at this 
time exceed the dynamic range of an input device such 
as a digital still camera. 

The converting table is determined from an 
exposure ratio, Rexp, of two images by a converting 
table preparation section. In the matrix circuit 23, 
a luminance signal value Y is obtained from R, G, B 
values. A luminance compression section 24 outputs 
a luminance value Y' which is compressed to adjust to 
the output device. Then, a ratio of compressed signal 




- 7 - 

to original one Y'/Y is obtained by a divider 25. 
The ratio Y'/Y is multiplied by outputs R, G, B of the 
linear converting section 22 by a multiplier 25 so as 
to be stored in the frame memory 16 as a joining image 
5 result. 

Generally, the signal value to be output from the 
imaging device is saturated for a certain amount of 
incident light in the case of longer time exposure as 
shown in FIG. 21. 
10 The value of an additional signal in which the 

signal of the longer exposure and that of the shorter 
exposure are added is changed with respect to the 
amount of incident light as shown by a bent line 
showing as an additional signal in FIG. 21. Then, 
15 a converting table preparation section 27 determines 

a table in which an amount of incident light I is 
estimated from an additional signal value S. 

Generally, since the image value is expressed by 
256 steps of 0 to 255, luminance Y of each pixel is 
20 compressed, for example by the following equation (1): 

Y ' = b ■ Y a ( 1) 

where a is a coefficient for determining a shape of 
compression and b is a coefficient for determining a 
gain of the entire image. 
25 If two different exposure images are synthesized 

to each other by the above-mentioned method, there can 
be obtained an image having the dynamic range almost 



equal to the film and can be seen well from a dark part 
to a bright part. 

However, in the conventional technique described 
in U.S. Patent 08/541,644, photographing conditions, 
which are necessary for correcting distortion, and the 
parameter of the optical system have to be set in 
advance. Due to this, it is difficult to get the image 
having high resolution image or a wide viewing angle 
image and a panorama image by simply using an arbitrary 
photographing device which the user has. 

The following will explain about the distortion 
with reference to FIGS. 23A, 23B, and 23C. 

More specifically, the distortion is generally a 
geometrical deformation, which is caused in accordance 
with the distance from the center of a lens. If the 
lattice object is photographed through the optical 
system without distortion, the image, which is shown 
in FIG. 23A, can be obtained. However, if the optical 
system suffered from distortion, the structure, which 
should be photographed by straight lines, are curved 
as shown in FIG. 23B. Thus, if the optical system 
has distortion, a straight line L is curved and 
captured as a curved line L' (FIG. 23C) . As a result, 
a point Y on the straight line L is moved to a point Y' 
on the curved line L' . In this case, an amount of 
distortion AR at one point on the image can be 
expressed by a polynomial expression (2): 
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AR = Al • R 3 + A2 • R 5 + (2) 
where R is a distance between a center of the image and 
the point Y. 

To get the image whose distortion is corrected, 
5 point Y' may be moved by only an amount of distortion 

AR on the straight line connecting the center C of the 
image to point Y' . However coefficients Al, A2 , 
differ depending on the focal position of the optical 
system, it is difficult for the general user to know 
10 coefficients Al, A2 , ••• correctly. 

Also, coefficients Al, A2 , •■■ differ depending on 
the apparatus to be used. Due to this, when the 
different apparatus is used, the correction coefficient 
must be adjusted again. 
15 Moreover, if images are taken by different zooming 

ratio, the size between adjacent images differs. Due 
to this, the images cannot be correctly joined to each 
other though the images are overlapped with each other. 
Also, for the object close to the user in such a case 
2 0 of an indoor place, the size is changed even if a 

photographer moves a few steps . 

Moreover, there is a case in which a white balance 
is automatically adjusted. For example, the color tone 
differs depending on a case in which the object is 
25 photographed in a direction toward or away from the sun. 

Due to this, when the images are joined by the 
above-mentioned technique, color is smoothly changed 
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but the entire image seems unnatural one. Moreover, in 
the technique of the wide dynamic range, the table for 
estimating the amount of incident light from the 
additional signal must be prepared as explained in the 
5 prior art. However, as shown in FIG. 21, a point N 

where the inclination of the additional signal is 
changed varies depending on the exposure ratio Rexp. 
For this reason, the user must know the exposure ratio 
Rexp of the plurality of images to be synthesized in 
10 advance. However, in many cases, the digital still 

cameras on the market have the structure in which the 
exposure can be adjusted but the user cannot know the 
ratio exactly. 

BRIEF SUMMARY OF THE INVENTION 
15 An object of the present invention is to provide 

an image processing apparatus which can correct images 
and be joined by a simple operation based on images 
themselves without knowing coefficients such as 
distortion of a camera, and an image processing 
20 apparatus which can effectively synthesize an image of 

a wide dynamic range image from only images . 

To achieved the above object, there is provided an 
image processing apparatus comprising: 

image input means for dividing one composition 
25 to have an overlap area where the same object as 

each other exists at a joining position to be input as 
a plurality of image parts; 
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correction parameter setting means for setting a 
correction parameter necessary to correct at least 
distortion of the plurality of image parts generated in 
each overlap area or a difference between the image 
5 parts; 

image correcting means for correcting at least one 
image part of the plurality of image parts in 
accordance with the set correction parameter to 
eliminate at least distortion of the plurality of image 

10 parts generated in each overlap area or the difference 

between the image parts ; 

image joining means for sequentially joining the 
plurality of image parts corrected by the image 
correction means in the overlap area to restore the one 

15 composition; and 

image display means for displaying the plurality 
of image parts input by the image input means, or at 
least one image part of the image parts corrected by 
the image correction means, or the restored image. 

2 0 According to the above-structured image processing 

device, since the image corrected by image correction 
means can be displayed on the display means to be 
confirmed, the image can be effectively corrected 
without knowing the necessary correction parameter, and 

25 an image correctly jointed by image joining means can 

be obtained. 

Also, according to the present invention, there is 
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provided 

an image processing device comprising: 

image input means for inputting one composition as 
a plurality of images photographed at a different 
5 exposure; 

correction parameter setting means for setting a 
correction parameter necessary to correct brightness of 
at least one image of the plurality of images having a 
different exposure; 
10 brightness correction means for correcting 

brightness of at least one image of the plurality of 
images in accordance with the set correction parameter; 

image display means for displaying at least one 
image of the images corrected by the brightness 
15 correction means; and 

joining means for estimating an amount of incident 
light obtained when the one input image is input based 
on the plurality of input images and the set correction 
parameter to convert the plurality of images whose 
2 0 brightness is corrected by the brightness correction 

means to be placed in a displaying range of the image 
display means, thereby joining the plurality of images. 

According to the above-structured image processing 
device, the images corrected by the image correction 
25 means are displayed on the image display means to set a 

correction parameter, the images whose brightness is 
corrected by the image joining means to be placed in 
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the displaying range of the image display means based 
on the correction parameter. 

The image processing device comprises correction 
parameter storing means for storing the correction 
5 parameters used in correcting the image in connection 

with names of corrected images or photographing devices 
and discrimination names of photographing methods. The 
correction parameter storing means selects a 
predetermined correction parameter from the correction 
10 parameters stored in the correction parameter storing 

means to be set . 

Moreover, the image processing device stores the 
parameter value once used in the correction in the 
correction parameter storing means, and the necessary 
15 value is selected from the correction parameter storing 

means. As a result, there is no need of newly 
correcting the parameter every image from the beginning. 
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
FIG. 1 is a view showing a schematic structure of 
20 an image processing device according to a first 

embodiment of the present invention; 

FIG. 2 is a view showing a specific structure of 
an image correction processing section of FIG. 1: 
FIG. 3A is a view showing a view showing 
25 instruction of processing to an image displayed on an 

image display section, and a correcting state of the 
image ; 
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FIG. 3B is a view showing instruction of 
processing to the image displayed on the image display 
section, and a correction menu; 

FIG. 4 is a view showing an example of a type of a 
5 file stored in a correction parameter storing section; 

FIG, 5A is a view showing composition of an image 
to be photographed; 

FIG. 5B is a view showing an example of the 
image photographed by an optical system having 
10 distortion; 

FIG- 5C is a view showing composition when joining 
the images photographed by the optical system having 
distortion; 

FIG. 6 is a showing a schematic structure of an 
15 image processing device according to a second 

embodiment of the present invention; 

FIG. 7 is a view showing the structure of the 
image correction processing section of FIG. 6; 
FIG. 8 is a view showing an example of a 
20 synchronizing result displayed on an image display 

section of the image correction processing section; 

FIG. 9 is a view showing the structure of an image 
correction processing section in an image processing 
device according to a third embodiment of the present 
25 invention; 

FIG. 10 is a view showing a relationship between a 
distance from the center of the image and a signal 
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value in order to explain peripheral reduction light; 

FIG. 11 is a view showing the structure of an 
image correction processing section in an image 
processing device according to a fourth embodiment of 
5 the present invention; 

FIG. 12 is a view showing an example of display to 
correct color display on the image display section of 
the image correction processing section; 

FIG. 13 is a view showing the structure Of an 
10 image correction processing section having an image 

expansion/reduction section and a correction parameter 
setting section; 

FIG. 14 is a view showing a schematic structure of 
an image processing device according to a fifth 
15 embodiment of the present invention; 

FIG. 15 is a view showing the structure of an 
exposure time ratio calculating section; 

FIG. 16 is a view showing the relationship between 
an input image signal before correction and the signal 
20 after correction in a brightness correcting section; 

FIG. 17 is a view showing a display example on the 
display section of brightness correction; 

FIG. 18 is a view showing an example of the 
structure of a conventional image processing device; 
25 FIG. 19 is a view showing an example of the 

structure of an image joining section of FIG. 18; 

FIG. 20 is a view showing an overlapping state of 



images at a wide angle image joining time to connect 
the images ; 

FIG. 21 is a view showing a characteristic of a 
signal obtained by adding a signal at long time 
exposure to a signal at short time exposure; 

FIG. 22 is a view showing one example of the 
structure of the image joining section of FIG. 18; 

FIG. 23A is a view showing an image having no 
curve; 

FIG. 23B is a view showing an image having 
distortion; and 

FIG. 23C is a view showing a characteristic of the 

distortion. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
Embodiments of the present invention will now be 
specifically described with reference to the 
accompanying drawings . 

FIG. 1 shows a schematic structure of an image 
processing apparatus according to a first embodiment of 
the present invention. This embodiment shows an 
apparatus for easily correcting distortion as viewing 
the display or so to synthesize the corrected images, 
thereby obtaining an image of high resolution and a 
wide angle image . 

The image processing apparatus of this embodiment 
comprises a memory card 32, a card reader 33, an image 
processing section 34, a monitor 7, a printer 8 for a 
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print output, and a storage device 9 such as an optical 
disk storing images, or a memory card. 

The memory card 32 records image data taken by a 
digital still camera 31 and a photographing condition 
5 data when the image was taken* The card reader 33 

reads out image data from the memory card 32. The 
image processing section 34 reproduces images from 
those image data, and provides correcting process such 
as distortion and a white balance to images to be 

10 joined. The monitor 7 displays the joined image and 

original image data . 

The image processing section 34 comprises an image 
data reproducing section 35, an image correction 
processing section 36, a signal switching section 37, 

15 and an image joining section 6 . 

The image data reproducing section 35 provides a 
processing such as a decompression to image data read 
by the card reader 33, and reproduces the images and 
photographing condition data. The image correction 

20 processing section 36 provides correcting process 

such as distortion and a color tone to the images. 
The signal switching section 37 executes an image 
switching for joining the images. 

Using the apparatus of the above-mentioned 

25 structure, a user takes an image to be divided such 

that parts of an object image are overlapped with 
each other. In other words, one composition is divided 
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to a plurality of image pieces to be taken such that 
the same object are captured at the end portion. 

Processing such as compression, addition of header 
data is provided to these photographed images (image a, 
5 image b, ••■ ) in the digital still camera 31. 

Thereafter, these images are recorded to the memory 
card 32 as image data. 

The memory card 32 is inserted to the card 
reader 33, and recorded image data is read out from the 

10 memory card 32 to be fetched to the image processing 

section 34. Fetched image data is input to the image 
data reproducing section 35, and processing such as 
decompression is provided thereto so that image data is 
reproduced. Then, the image correction processing 

15 section 36 provides a correcting process, and corrected 

image data is input to the image joining section 6. 
The image joining section 6 has the structure as shown 
in FIG. 19. The image joining section 6 provides the 
same processing as described in U.S. Patent 08/045,038 

20 to join the images. The joined image is output to the 

monitor 7, the printer 8, or the storage medium 9. 

FIG. 2 explains the specific structure of the 
image correction processing section 36. 

The image correction processing section 36 

25 comprises an image display section 44, a distortion 

correction processing section 41, a correction 
parameter storage section 42, and a correction 
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parameter setting section 43. 

The image display section 44 displays an original 
image and a corrected image. The distortion correction 
processing section 41 provides a distortion to the 
5 input image. The correction parameter storage section 

42 stores the parameter, which is used in the 
correcting process due to the distortion correction 
processing section 41. The correction parameter 
setting section 43 adjusts the correction parameter to 

10 be set by the user's operation. Or, the correction 

parameter setting section 4 3 selects the correction 
parameter read out from the correction parameter 
storage section 42 to be set. 

The image correction processing section 36 

15 differently works depending on the case. There are two 

cases, one case is that a distortion correction is 
provided to images taken by a certain camera in which 
distortion correction is never provided to its images, 
which is used for the first time. The other case is 

20 that the correction is provided to images taken with a 

camera in which distortion correction has been already 
provided to its images and correction history has been 
left. 

First, the following will explain the case in 
25 which the distortion correction has not been provided 

to the images taken by a camera. 

The distortion correction processing section 41 
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inputs an image (e.g., image a) having distortion as 
shown in FIG. 23B. Then, the distortion correction 
processing section 41 outputs the image corrected based 
on equation (2) to the image display section 44 by use 
5 of an initial value, set in advance, of coefficients Al, 

A2, in equation (2). The image display section 44 
simultaneously displays the corrected image and an 
original image a. 

Then, when a desired coefficient is set by the 

10 user's operation, the correction parameter setting 

section 43, which comprises a mechanism for adjusting 
coefficients Al, A2 , feeds back new coefficients Al, 

A2, *• to the distortion correction processing section 
41 to renew the coefficients immediately* 

15 The distortion correction processing section 41 outputs 

the image corrected by the new coefficient set, and the 
image display section 44 changes the displayed image to 
an image newly corrected. 

The user operates the correction parameter setting 

20 section 43 as viewing at least one of two images. 

Thereby, coefficients Al, A2 , , which are used for 
appropriately correcting the image, are determined. 
At this time, to make the user operates easily, the 
correction parameter setting section 4 3 is preferably 

25 structured such that an imaginary adjusting knob 

displayed on the display section 44 is operated by a 
mouse, and a keyboard as shown in FIG. 3A. If the 
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coefficients (Al, A2 in FIGS. 3A, 3B) are changed by 
the operation of the knob f the "image after correction" 
on the screen is recorrected in accordance with the new 
coefficient value, so that the image is renewed. If 
5 two or more images are needed to be displayed on the 

display screen, an arbitrary number of images can be, 
of course, displayed. 

When the user judges that the correction is 
sufficient, he/she presses an "OK" key (clicks in 

10 the case of the mouse) to determine the coefficients Al, 

A2, Thus, the determined coefficients are stored in 

the correction parameter storage section 42 together 
with names identifying cameras, which are arbitrarily 
added by the user. For example, they may be stored to 

15 the correction parameter storage section 42 as a file 

as shown in FIG. 4. 

Arbitrary images can be used for adjustment of 
the coefficients. However, the image which includes 
abject with straight lines such as a building, a book 

20 shelf, a window frame, and graph paper, can be easily 

adjusted. 

Next, the following will explain for the case 
of correcting images by the camera in which its images 
has been already corrected in the past and the 
25 coefficients Al, A2, •■■ by that time are stored in the 

parameter storage section 42 as a file. 

In this case, since the correction parameters are 



stored with names of photographing equipments, those 
data is read when the image processing device is 
started, and displayed on the image display section 44 
in a menu form as shown in FIG. 3B. If the user 
selects the name of the device to be used from the menu 
screen, the corresponding coefficients Al, A2, ••• are 
read out to the distortion correction processing 
section 41 from the correction parameter storage 
section 42 through the correction parameter setting 
section 43, Thereby, the image to which the distortion 
correction processing is provided is output from the 
distortion correction processing section 41. 

As explained above, according to this embodiment, 
it is unnecessary to know distortion data of the 
optical system of photographing devices in advance. 
In other words, the user can determine the correction 
coefficients as viewing only the image itself. As a 
result, even in the case of the camera which is used 
for the first time, there is no need of knowing the 
characteristic of the optical system in advance. 

Also, in the case of using the camera in which the 
photographed images were corrected in the past, the 
correction coefficients of distortion are recorded as 
data. As a result, the correction processing can be 
executed by only selecting data from the menu. 
Therefore, it is possible to avoid the complication of 
the operation in which the parameter must be set every 
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time to correct the entire images used in the image 
synchronization. 

In this embodiment, there was used the digital 
still camera in which the images are stored in the 
5 memory card. However, the present invention is not 

limited to the digital still camera. The image 
processing may be executed even by a device in which 
data is directly input to the image reproducing section. 
Moreover, the above embodiment explained the case in 
10 which the plurality of images was photographed by one 

input device. However, the images can be photographed 
by the structure using a plurality of input devices 
simultaneously. Furthermore, though this embodiment 
explained that the correction parameter setting section 
15 43 was the imaginary adjusting knob on the image 

display section 44, another switch of a rotation type 
or a slide type may be used. 

Moreover, this embodiment explained the structure 
in which the image was corrected as comparing the 
20 original image with the processing result. However, 

in the case of the image of the object which includes 
straight line structure such as a building, a book 
shelf, a window frame, etc., only the image after the 
correction processing is displayed on the image display 
25 section 44, and the lines are adjusted to be straight. 

Thereby, the coefficients Al , A2 can be determined. 

The following will explain the image processing 



device of a second embodiment. 

This embodiment, which is a modification of the 
first embodiment, will be explained with reference to 
FIGS. 5 to 8. In these figures, the same reference 
numerals are added to the same structural portions as 
FIGS- 1 and 2, and the explanation is omitted. 

FIG. 6 is a view showing the structure of this 
embodiment. The second embodiment is different from 
the first embodiment in the point that the joined image 
output from the image joining section 6 is input to the 
image correction processing section 36. FIG. 7 shows 
the structure of the image correction processing 
section 36. FIG. 8 shows one example of the image 
displayed on the image display section 44 of the image 
correction processing section 36. 

According to this structure, image data (image a) 
to which decompression is provided by the image data 
reproducing section 35 is input to the distortion 
correction processing section 41 so that data, which is 
corrected by correction parameter values Al, A2, set 
in advance, is output to the image joining section 6. 
Similarly, the images, which are adjacent to the 
corrected image, are corrected by the distortion 
correction processing section to be output. The output 
images are joined by the image joining section 6, so 
that the joined image is generated. Then, the 
resultant image is input to the image correction 
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processing section 36 , and displayed by the image 
display section 44. 

Next, the following will explain an example of the 
actual processing by the user's operation with 
5 reference to FIGS- 5A, 5B, and 5C. 

FIG. 5A is an object to be photographed. When 
composition of the object is divided and input through 
the optical system having distortion to have an overlap 
area where the same object as each other exists at a 

10 joining position as a plurality of image parts. 

For example, a plurality of pairs of 
characteristic points (PI and PI', P2 and P2 ' , and P3 
and P3' in FIG. 3B) are set on the image. This setting 
can be automatically executed by the image joining 

15 section 6 under control of a predetermined program. 

Or, this setting can be designated by the user's 
operation. Two points are selected from the pair of 
characteristic points. The following will explain the 
case in which PI and P2 are selected. 

20 At this time, it is assumed that an amount of 

parallel movement and an amount of rotation are 
obtained on the basis of PI and P2 and the images are 
joined by the image joining section 6 as the distortion 
correction coefficients Al, A2 , are maintained 

25 incorrectly as they are. As a result, as shown in 

FIG. 5C, PI and Pl f are correctly conformed to each 
other. Also, P2 and P2 ' are correctly conformed to 



each other. However, P3 and P3' are not conformed to 
each other. Then, the other points of its surroundings 
are not conformed to each other. 

At this time, the joined result is displayed on 
the image display section 44, and the user adjusts the 
coefficients Al, A2 , ■•• such that points other than 
reference points PI and P2 are conformed to each other 
as viewing the display by the image processing device 
shown in FIG. 6. The adjusted coefficients Al, A2 , 
are immediately input to the distortion correction 
processing section 41. Then, the newly corrected 
images are joined to each other by the image joining 
section 6, and displayed on the image display section 
44 again. 

At this time, if the coefficients Al , A2 , ** are 
correctly set, not only Pi and PI', P2 and P2 ' but also 
P3 and P3' are conformed to each other, simultaneously. 
By this method, the image whose distortion is seemingly 
unclear such as a landscape image, and a figure image 
can be easily corrected. 

In the above embodiment, the user adjusted the 
coefficients Al, A2 , ••- as viewing the images 
displayed on the image display section 44. However, 
a shift between P3 and P3' is automatically detected, 
and the coefficients Al, A2, ■■• may be automatically 
corrected such that the detected shift becomes 0 or 
minimum . 
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Next, FIG. 9 shows the structure of the image 
correction processing section of the image processing 
device of a third embodiment. 

In these figures, the same reference numerals are 
5 added to the same structural portions as the first and 

second embodiments, and the explanation is omitted. 

Generally, due to influence of peripheral 
reduction light in the optical system such as a camera, 
the brightness of the images is reduced as the image 
10 advances to the periphery. As a result, when the 

images with the peripheral reduction light are joined 
to each other, the image becomes dark at the 
overlapping area, and an unnatural joining image is 
generated. 

15 The peripheral reduction light is a phenomenon in 

which the brightness of the image becomes darker as a 
distance R from the center of the image is increased as 
shown in FIG. 10. A ratio of signal value S' to an 
ideal signal value with the peripheral reduction light 

20 can be approximately obtained by the following 

polynomial expression (3): 

S'/S = BO + Bl • R + B2 • R 2 + '* (3) 
In this case, the image correction processing 
section 36 comprises the image display section 44, a 

25 peripheral reduction light correction processing 

section 46, a peripheral reduction light correction 
parameter storage section 47, and a peripheral 
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reduction light correction parameter setting selection 
section 48. 

In the peripheral reduction light correction 
parameter setting selection section 48, an imaginary 
5 knob for displaying coefficients BO, Bl, B2 , is moved 

to set such that brightness of the image center and 
that of the surroundings are the same. 

If the coefficients BO, Bl, B2 , are set by the 
above-mentioned structure, the image can be easily 

10 corrected based on the image itself even if the user 

does not know various parameters of the optical system 
of a camera. As a result, the images joined by the 
image joining section 6 does not become dark even at 
the overlapping portion, and a natural image can be 

15 obtained as an entire screen. 

Same as the case of distortion, the marginal is 
the phenomenon peculiar to its optical system of the 
imaging apparatus. As explained in the first 
embodiment, the correcting coefficient values are once 

20 stored in accordance with names of photographing 

devices. Then, the images can be corrected by 
selecting the name from the menu at the second time and 
the following. 

Moreover, this embodiment explained that the 

25 images were corrected as comparing the original image 

with the process result. However, since the brightness 
of the entire image may be uniformed, only the image 



after correction is displayed on the image display 
section 44, and the coefficients BO, Bl, B2, 1 may be 
set and selected. 

Next, the following will explain the structure for 
correcting a color tone by the image processing 
apparatus of a fourth embodiment of the present 
invention with reference to FIGS. 11 and 12. 

The image processing correction processing section 
36 in FIG. 11 comprises the image display section 44, a 
color tone correcting section 51, and the correction 
parameter setting section 43. 

The color tone correcting section 51 converts 
the original image to hue (H), saturation (S), and 
intensity (I), respectively by HIS transformation. 
The correction parameter setting section 4 3 adjusts the 
correction parameter for transformation to H, S, I so 
as to correct the color tone. 

The color correction section 51 corrects the input 
image based on hue H, saturation S, and intensity I to 
be output to the image display section 44. The user 
operates the knob for adjusting hue H, saturation S, 
and intensity I on the screen shown in FIG. 12 as 
comparing the images displayed on the image display 
section 44, in order that both images have the same 

color tone. 

By the above-structured image correction 
processing device 36, not only the color tone of the 
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overlapping area where the adjacent images are joined 
but also the color tone of the overall of the image 
can become natural. This embodiment explained that 
the original image was converted to hue (H) , saturation 
5 (S), and intensity (I), respectively. However, similar 

to the adjustment of white balance, signal levels of R, 
G, B can be adjusted. 

Moreover, as shown in FIG. 13, an image 
magnification/reduction section 52 and the correction 
10 parameter setting selection section 43 are provided to 

the image correction processing section 36. Thereby, 
there can be obtained a well joined image in accordance 
with the difference in a zoom ratio, and the change of 
the object size due to the change of the photographer's 

15 position. 

Next, the following will explain the structure to 
extend the dynamic range of the input image by the 
image processing apparatus of a fifth embodiment of the 
present invention with reference to FIGS. 14 to 17. 

20 The image processing apparatus of this embodiment 

calculates exposure ratio Rexp from the taken image, 
and synthesizes images with the calculated exposure 
time ratio. In these figures, the same reference 
numerals are added to the same structural portions as 

25 the first embodiment, and the explanation is omitted. 

The image processing apparatus of this embodiment 
has the structure in which an exposure time ratio 
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calculating section 61 is provided to the image 
processing section 34. 

In the image processing section 34, reproduced 
images a and b are input to the exposure time ratio 
5 calculating section 61 through the signal switching 

section 37. As a result, an exposure time ratio Rexp 
is input to the image joining section 6. In the image 
joining section 6, the images a and b are synthesized 
with the calculated exposure time ratio Rexp, and the 
10 output image exceeds the dynamic range of the input 

device. Since the structure of the image joining 
section 6 uses the structure same as FIG. 22, the 
explanation is omitted. 

FIG. 15 explains the structure of the exposure 
15 time ratio calculating section 61. 

The exposure time ratio calculating section 61 
comprises the image display section 44, an exposure 
time ratio setting section 6 3 for adjusting the 
exposure time ratio Rexp by the user, and a brightness 
20 correcting section 62 for correcting brightness of the 

input image based on the exposure time ratio Rexp set 
by the exposure time ratio setting section 63. 

FIG. 16 is a view showing the processing by the 
brightness correcting section 62. The signal Sout, 
25 which is obtained by correcting the image signal Sin, 

can be expressed by the following equation (4): 
Sout = Rexp • Sin " ( 4 ) 
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At this time, if Rexp > 1.0, the image becomes 
brighten. If Rexp < 1.0, the image becomes darker. 

FIG. 17 shows a display example on the display 
section. 

5 In the image a, which is used as a reference image, 

the indoor portion where a table, a flower, etc are 
presented is captured at a suitable exposure. Then, 
the portion out of the window where woods and mountains 
can be seen is captured at an over-exposure. In the 
10 image b, which is used as an adjusting image, the 

portion out of the window is formed at the suitable 
exposure, and the indoor portion is formed at the 
under-exposure . 

The image a is displayed as a reference image, and 
15 the knob displayed on the screen is adjusted to correct 

brightness of the image b. The user designates the end 
of adjustment with an "OK" key when the brightness of 
the entire image b reaches the same as the reference 
image (image a). The exposure time ratio Rexp set in 
20 this way is output to the image joining section 6, and 

the processing of the wide dynamic range is executed. 

According to this embodiment, since the user can 
obtain exposure time ratio Rexp as confirming the image, 
a desired device can be used in the input device for 
25 inputting the image. Also, since it is unnecessary to 

record exposure time when taking images, the processing 
of the wide dynamic range can be easily executed. 



- 33 - 



10 



Moreover, this embodiment explained the example in 
which the exposure time ratio between two images was 
used as a correction parameter. However, the present 
invention is not limited to the above example. The 
exposure time ratio among three or more images can be 
used as a correction parameter. The embodiment 
explained that one image was used as a reference image 
and the brightness of only the other image was 
corrected. However, brightness of both images can be 
corrected to generate an image having intermediate 
brightness. At this time, the exposure time ratio can 
be shown as in FIG . 16. 

Moreover, as a method for photographing a 
plurality of brighter and darker images having a 
15 different exposure, the diaphragm of the camera can be 

changed, and the amount of incident light may be 
controlled by transmittance of the reduction light 
filter. In other words, as the correction parameter, 
which is necessary for the processing of the wide 
20 dynamic range, the parameter, which shows the exposure 

ratio among the plurality of images, may be used. 
For example, the exposure time ratio shown in the above 
embodiment, a diaphragm ratio, a transmittance ratio of 
the reduction light filter can be used as a correction 

2 5 parameter . 

In the above-explained embodiments, the images are 
taken by the digital still camera. However, the same 
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processing as in the embodiments can be executed even 
in the image, which is obtained by digitizing the film 
photographed by the conventional camera, or the image, 
which is input from the image photographed by the video 
5 camera or the digital video camera through an image 

input board. The monitor 7, which outputs the joined 
image, can be used as the image display device 44 of 
each embodiment . 

Moreover, the above-explained embodiments can be 

10 combined with each other. Particularly, the correction 

parameter storage section 42 can be combined with the 
structure described in the first to third embodiments. 
In this case, the correction parameter storage 
section 42 can store the parameter if the user knows 

15 the characteristic of the optical system. Also, the 

correction parameter storage section 42 can be combined 
with the structure described in the fourth and fifth 
embodiments. In this case, the correction parameter 
storage section 42 can store the parameter to be used 

20 as a default value when the user executes the 

adjustment . 

According to the present invention, there can be 
provided the image processing apparatus, which can 
correct only from the image photographed by a various 
25 imaging apparatus which the user has or the 

photographing device of a reasonable price, thereby 
obtaining the joined images with wide viewing angle and 
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the image of the wide dynamic range. In this case, 
there is no need of knowing the characteristics of the 
distortion and peripheral reduction light optical 
system in advance. Moreover, there is no need of using 
5 the photographing device of a high price, which can 

control and record the photographing conditions such as 
exposure time f white balance, etc. 
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CLAIMS 

1. An image processing apparatus comprising: 
image input means for getting a plurality of image 

parts dividing one composition such that the image 
5 parts have overlapping areas , each having the same 

image of an object in the overlapping area as in the 
overlapping area of the next image part; 

correction parameter setting means for setting a 
correction parameters necessary to correct at least 
10 distortion of said plurality of image parts generated 

in each overlap area or a difference between the image 
parts ; 

image correcting means for correcting at least one 
image part of said plurality of image parts in 
15 accordance with said set correction parameter to 

eliminate at least distortion of said plurality of 
image parts generated in each overlap area or the 
difference between the image parts; 

image joining means for sequentially joining the 
20 plurality of image parts corrected by said image 

correction means in said overlap area to restore said 
one composition; and 

image display means for display at least said 
plurality of image parts input by said image input 
25 means or said image parts corrected by said image 

correction means . 

2. An image processing apparatus comprising: 



image input means for inputting one composition as 
a plurality of images taken with a different exposure; 

correction parameter setting means for setting 
correction parameters necessary to correct brightness 
of at least one image of said plurality of images 
having a different exposure; 

brightness correcting means for correcting 
brightness of at least one image of said plurality of 
images in accordance with said set correction 
parameters ; 

image display means for displaying at least one 
image of the images corrected by said brightness 
correction means; and 

image synthesizing means for estimating an amount 
of incident light obtained when said one input image is 
input based on said plurality of input images and said 
set correction parameters to convert said plurality of 
images whose brightness is corrected by said brightness 
correction means to be placed in a displaying range of 
said image display means, thereby joining said 
plurality of images. 

3. The image processing apparatus according to 
claim 1 or 2, wherein said image correction means 
corrects the image by changing the correction parameter 
in accordance with differences in brightness between a 
plurality of images displayed by said image display 
means . 



4. The image processing apparatus according to 
claim 1 or 2, wherein said image correction means 
corrects the image by changing the correction parameter 
in accordance with the distortion of one image 
displayed by said image display means or in accordance 
with differences in distortion between a plurality of 
images displayed by said image display means. 

5. The image processing apparatus according to 
claim 1 or 2, wherein said image correcting means 
corrects the image by changing the correction parameter 
in accordance with differences in image magnification 
between a plurality of images displayed by said image 
display means . 

6. The image processing apparatus according to 
claim 1 or 2, wherein said image correcting means 
corrects the image by changing the correction parameter 
in accordance with differences in color data between a 
plurality of images displayed by said image display 
means . 

7. The image processing apparatus according to 
claim 6, wherein said color data is at least one of hue, 
saturation, and intensity. 

8. The image processing apparatus according to 
claim 6, wherein said color data is at least one of R, 
G and B values for adjusting a white balance. 

9. The image processing apparatus according to 
claim 1 or 2, wherein said image correction means 



corrects the image by changing the correction parameter 
in accordance with peripheral reduction light of one 
image display by said image display means or in 
accordance with differences in peripheral reduction 
light between a plurality of images displayed by said 
image display means. 

10. The image processing apparatus according to 
claim 3, wherein said image correction means corrects 
the image by changing an exposure ratio between a 
plurality of images, which is used as said correction 
parameter, in accordance with difference in brightness 
between said plurality of images displayed by said 
image display means . 

11. The image processing apparatus according to 
claim 4, wherein further comprising correction 
parameter storing means for storing one or a plurality 
sets of said correction parameters used in correcting 
said image in connection with the name of the imaging 
apparatus used to take the image, and said correction 
parameter setting means selects a desired set of 
correction parameters from the correction parameters 
stored in said correction parameter storing means . 

12. The image processing apparatus according to 
claim 9, wherein further comprising correction 
parameter storing means for storing one or a plurality 
sets of said correction parameters used in correcting 
said image in connection with the name of the imaging 



apparatus used to take the image , and said correction 
parameter setting means selects a desired set of 
correction parameters from the correction parameters 
stored in said correction parameter storing means. 
13. An image processing method comprising: 
an image input step of getting a plurality of 
image parts dividing one composition such that the 
image parts have overlapping areas, each having the 
same image of an object in the overlapping area as in 
the overlapping area of the next image part; 

a correction parameter setting step of setting 
correction parameters necessary to for correct at least 
images distortion or image difference occurring in the 
overlapping areas of each image part; 

an image correcting step of correcting at least 
one of said plurality of image parts in accordance with 
said correction parameters, thereby to correct 
distortion of images or image difference occurring in 
at least the overlapping areas of each image part; 

a composition restoring step of restoring said 
composition by sequentially combining said plurality of 
image parts corrected, one to another, with overlapping 
the same at overlapping areas; and 

an image displaying step of display at least said 
plurality of image parts input or said plurality of 
image parts corrected. 

14. An image processing method comprising: 



- 41 - 

an image input step of inputting a plurality of 
images obtained by taking one composition at different 
exposures ; 

a correction parameter setting step of setting a 
5 correction parameters indispensable for correcting the 

brightness of at least one of said plurality of images 
taken with different exposures; 

an image correcting step of correcting the 
brightness of said at least one image in accordance 
10 with the correction parameter set; 

an image displaying step of displaying at least 
one of images corrected in the image correcting step; 
and 

an image synthesizing step of combining said 
15 plurality of images corrected in brightness in the 

image correcting step, into one image to be displayed 
within a range of the imaging display step, by 
inferring an amount of incident light obtained when 
said composition is input in the image input step, from 
20 said plurality of images which have been input and said 

correction parameter which has been set. 

15. The image processing method according to 
claim 13 or 14, which further comprises a correction 
parameter storing step of storing one or a plurality 
25 sets of the correction parameters used in correcting 

the image, in connection with the name of the 
photographing apparatus and the name of the 
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photographing method used to photograph the image, and 
in which said correction parameter setting step is to 
select and set a desired set of correction parameters 
from correction parameters stored in said correction 
5 parameter storing step. 

16. The image processing method according to 
claim 13 or 14, which further comprises a correction 
parameter storing step of storing one or a plurality 
sets of the correction parameters used in correcting 

10 the image, in connection with the name of the 

photographing apparatus and the name of the 
photographing method used to photograph the image, and 
in which said correction parameter setting step is to 
select and set a desired set of correction parameters 

15 from correction parameters stored in said correction 

parameter storing step. 

17. The image processing method according to 
claim 13 or 14, wherein said image correcting step is 
to change the correction parameter in accordance with 

20 differences in image magnification between a plurality 

of images displayed in said image displaying step. 

18. The image processing method according to 
claim 13 or 14, wherein said image correcting step is 
to correct the image by changing the correction 

25 parameter in accordance with differences in color data 

between a plurality of images displayed by said image 
display means . 
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19. The image processing method according to 
claim 13 or 14, wherein said image correcting step is 
to is to correct the image by changing the correction 
parameters in accordance with peripheral reduction 
light of one image displayed in said image displaying 
step or in accordance with differences in peripheral 
reduction light between a plurality of images displayed 
in said image displaying step. 

20. The image processing method according to 
claim 13 or 14, wherein said image correcting step is 
to correct the image by changing an exposure ratio 
between a plurality of images, which is used as said 
correction parameter, in accordance with differences in 
brightness between said plurality of images displayed 

15 in said image displaying step. 

21. A recording medium recording computer programs 
for restoring an image by combining a plurality of 
image parts divided from one composition, each image 
parts having the same image of an object in an 

20 overlapping area, said recording medium recording: 

an image inputting program for inputting said 
plurality of image parts; 

a correction parameter setting program for setting 
correction parameters indispensable for correcting 
25 images distortion or image difference occurring in at 

least the overlapping areas of each image part; 
an image correcting program for correcting 



at least one of said plurality of image parts in 
accordance with said correction parameters, thereby to 
correct distortion of images or image difference 
occurring in at least the overlapping areas of each 
image part; 

a composition restoring program for restoring said 
composition by sequentially combining said plurality of 
image parts corrected, one to another, with overlapping 
the same at overlapping areas; and 

an image displaying program for displaying said 
plurality of image parts input, or at least one of said 
plurality of image parts corrected. 

22. A recording medium recording computer programs 
for correcting a plurality of images obtained by taking 
one composition with different exposures, to provide an 
image having a desired brightness, said recording 
medium comprising: 

an image inputting program for inputting one 
composition in the form of a plurality of images 
photographed at different exposures; 

a correction parameter setting program for setting 
a correction parameters indispensable for correcting 
the brightness of at least one of said plurality taken 
with images photographed at different exposures; 

an image correcting program for correcting the 
brightness of said at least one image in accordance 
with the correction parameter set; 



an image displaying program for displaying at 
least one of images corrected in accordance with the 
image correcting program; and 

an image synthesizing program for combining said 
plurality of images corrected in brightness in 
accordance with said image correcting program, into one 
image to be displayed within a range of accordance with 
said imaging display program, by inferring an amount of 
incident light obtained when said composition is input 
in accordance with said image inputting program, from 
said plurality of images which have been input and said 
correction parameter which has been set. 

23. The recording medium according to claim 21 
or 22, which further comprises a correction parameter 
storing program for storing the correction parameter 
used in correcting the image, in connection with the 
name of the photographing apparatus and the name of the 
photographing method used to photograph the image, and 
in which said correction parameter setting program is 
designed to select and set a desired correction 
parameter from correction parameters stored in 
accordance with said correction parameter storing 
program. 

24. The recording medium according to claim 21 
or 22, which further comprises a correction parameter 
storing program for storing the correction parameter 
used in correcting the image, in connection with the 
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name of the photographing apparatus and the name of the 
photographing method used to photograph the image, and 
in which said correction parameter setting program is 
designed to select and set a desired correction 
5 parameter from correction parameters stored in 

accordance with said correction parameter storing 
program. 

25. The recording medium according to claim 21 
or 22, wherein said image correcting program is 
10 designed to correct the image by changing the 

correction parameter in accordance with differences in 
image magnification between a plurality of images 
displayed in accordance with said image displaying 
program, 

15 26. The recording medium according to claim 21 

or 22, wherein said image correcting program is 
designed to correct the image by changing the 
correction parameter in accordance with differences in 
color data between a plurality of images displayed in 

20 accordance with said image display program. 

27. The recording medium according to claim 21 
or 22, wherein said image correcting program is 
designed to correct the image by changing the 
correction parameter in accordance with peripheral 

25 reduction light of one image displayed by using said 

image displaying program, or in accordance with 
differences in peripheral reduction light between a 



plurality of images displayed in said image displaying 
step. 

28. The recording medium according to claim 21 
or 22, wherein said image correcting program is 
designed to correct the image by changing an exposure 
ratio between a plurality of images, which is used as 
said correction parameter, in accordance with 
differences in brightness between said plurality of 
images displayed by using said image displaying program. 
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ABSTRACT OF THE DISCLOSURE 
In an image processing apparatus of the present 
invention, which displays image data, such as images 
photographed by imaging apparatus such as a digital 
5 camera and photographing conditions as images before 

correction and images after correction, on an image 
display section, distortion and color tone of images 
obtained from an unused camera are corrected and joined 
to each other as viewing images before and after 
10 correction, so that the parameters used in the 

correction are stored in accordance with its apparatus, 
and the stored parameters are read out and used to 
correct in images obtained from previously used camera, 
and the joined image and image data are displayed on a 
15 monitor and output by a printer, or stored in a 

recording medium. 
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DECLARATION FOR PATENT APPLICATION (97S0797) 

As a below named inventor, I declare that my residence, post office address and citizenship are as stated below above my 
name- 1 believe that I am the original, first and sole inventor (if only one name is listed below) or an original, first and joint 
inventor (if plural names are listed below) of the subject matter which is claimed and for which a patent is sought on the 
invention entitled: 

IMAGE PROCESSING APPARATUS 

the specification of which (check applicable box(es)). 

[X] is attached hereto. , A t - 

[ ] was filed on as United States Application No. or PCT International Apphcation 

No. 

f 1 and was amended on . , . 

I hereby state that I have reviewed and understand the contents of the above-idenUfied specification, including the claims, 

as amended by any amendment referred to above. . 
I acknowledge the duty to disclose information which is material to patentability as defined m 37 CFR 1.56. 
I hereby claim foreign priority benefits under 35 U.S.C. H9(a)-(d) or 365(b) of any foreign apphcation(s) for patent or 
inventor's certificate, or 35 U.S.C. 365(a) of any PCT International application which designated at least one country other 
than the United States, listed below and have also identified below any foreign apphcation for patent or inventor s certi- 
ficate or PCT International apphcation having a filing date before that of the application on which priority is claimed: 
'Japanese Patent Apphcation No. 8-296750, filed November 8, 1996 Priority Claimed 

Japanese Patent Apphcation No. 8-312260, filed November 22, 1996 Priority Claimed 

I hereby claim the benefit under 35 U.S.C. 119(e) of any United States provisional application(s) listed below. 

I hereby claim the benefit under 35 U.S.C. 120 of any United States application(s) or 35 U.S.C. 365(c) of any PCT 
International application designating the United States, listed below and, insofar as the subject matter of each of the 
claims of this apphcation is not disclosed in the prior United States apphcation or PCT International apphcation m the 
manner provided by the first paragraph of 35 U.S.C. 112, I acknowledge the duty to disclose information which is 
material to patentability as defined in 37 CFR 1.56, which became available between the filing date of the prior apph- 
cation and the national or PCT international filing date of this application. 

I hereby appoint as my attorneys, with full powers of substitution and revocation, to prosecute this apphcation and transact 
all business in the Patent and Trademark Office connected therewith: Stephen H. Frishauf (Reg ^Na 16,232) ^onaxd 
Holtz (Reg. No. 22,974), Herbert H. Goodman (Reg. No. 17,081), Thomas Langer (Reg. No. 27,264) MarshaU J. Chick 
fRets No 26 853), Walter J. Baum (Reg. No. 20,641), Richard S. Barth (Reg. No. 28,180) and Douglas Holtz (Reg. No 
33902) each of whose address is 767 Third Avenue - 25th Floor, New York, N.Y. 10017-2023, or any one of them and 
request that correspondence be directed to Frishauf, Holtz, Goodman, Langer &. Chick, P.C., 767 Third Avenue - 25th 
Floor, New York, N.Y. 10017-2023. 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information 
and belief are believed to be true; and further that these statements were made with the knowledge that wdlful fake state- 
ments and the like so made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United 
States Code and that such willful fake statements may jeopardize the validity of the apphcation or any patent issued thereon. 

Pos/offu-rAddress: c/o Intellectual Property & Legal Department, OLYMPUS OPTICAL CO., LTD., 2-3, 

Kuboyama-cho, Hachioji-shi, Tokyo, Japan 
Residence Address: Hino-shi, Tokyo, Japan 
Country of Citizenship: Japan 

nat e: October 27, 1997 T^UvlpJ<U ftU^UrtcM-U 

Masaki Higurashi 

Post Office Address: c/o Intellectual Property & Legal Department, OLYMPUS OPTICAL CO., LTD., 2-3, 

Kuboyama-cho, Hachioji-shi, Tokyo, Japan 
Residence Address: Hachioji-shi, Tokyo, Japan 
Country of Citizenship: Japan 



Date: October 27. 1997 . 

Yasuhiro Komiya 



flhiro Komiya 
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[3rd Inventor] 

Post Office Address: c/o Intellectual Property & Legal Department, OLYMPUS OPTICAL CO., LTD., 2-3, 

Kuboyama-cho, Hachioji-shi, Tokyo, Japan 
Residence Address: Hachioji-shi, Tokyo, Japan 
Country of Citizenship: China 



Date: October 27, 1997 




GangLu Jing 



